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Abstract 

Urn models have been widely studied and applied in both scientific and social disci- 
plines. In clinical studies, the adoption of urn models in treatment allocation schemes has 
been proved to be beneficial to both researchers, by providing more efficient clinical trials, 
and patients, by increasing the probability of receiving the better treatment. In this paper, 
we endeavor to derive a very general class of immigrated urn models that incorporates 
the immigration mechanism into the urn process. Important asymptotic properties are 
developed and illustrative examples are provided to demonstrate the applicability of our 
proposed class of urn models. In general, the immigrated urn model has smaller variability 
than the corresponding urn model. Therefore, it is more powerful when used in clinical 
trials. 
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1 Introduction 



1.1. Urn Models and their applications 

Urn models have long been considered powerful mathematical instruments in many 
areas, including the physical sciences, biological sciences, social sciences, and engineering 
(Johnson and Kotz, 1977; Kotz and Balakrishnan, 1997). For example, in medical science, 
Knoblauch, Neitz, and Neitz (2006) apply an urn model to study cone ratios in human 
and macaque retinas. In population genetics, Benai'm, Schrciber, and Tarres (2004) make 
use of a class of generalized Polya urn models to scrutinize evolutionary processes. In 
economics, the model is employed to capture the mechanism of reinforcement learning 
by Erev and Roth (1998) and Beggs (2005). In business, a coupled Polya urn model is 
utilized by Windrum (2004) to examine the recent browser war between Netscape and 
Microsoft. In decision science, McCabe-Dansted and Slinko (2006) employ the urn model 
to investigate the social choice rules that are influential factors which affect the outcomes 
of elections. In addition, numerous examples of applications of urn models in the areas 
of physics, communication theory, and computer science are also provided by Milenkovic 
and Compton (2004). 

In recent decades, the application of urn models in clinical trials has attracted great 
attention from biomedical researchers. Urn models have played an important role in the 

advancement of adaptive design methodologies that provide treatment allocation schemes 
which depend on the history of treatment outcomes. The principle of adaptive designs 
is to skew the probability of treatment allocation so that fewer patients receive inferior 
treatment, amid the accumulation of information on treatment effectiveness as the clinical 
study progresses. 

In adaptive design, an early influential urn model is the generalized Polya urn (GPU) 
that was discussed in Athreya and Karlin (1968), Wei and Durham (1978), and Wei (1979). 
The GPU is based on the Polya urn model which was originally constructed to model 
contagious diseases (Eggenberger and Polya, 1923). For example, the GPU model of Wei 
(1979) can be described as following: Assume that each incoming patient is assigned to 
one of K treatments, and the GPU is updated at each stage by adding one additional ball 
of the same treatment type if the treatment is successful, and by adding an additional ■j^:^ 
ball of the other K —1 treatment types if the treatment is unsuccessful. By embedding the 
urn process in a branching process, asymptotic properties of the GPU model have been 
obtained in Janson (2004). 

Another class of urn models with applications to clinical trials is the ternary urn 
model (Ivanova and Flournoy, 2001). A special case is the randomized Polya urn (RPU) 
proposed by Durham, Flournoy, and Li (1998). Unlike the GPU of Wei (1979), the RPU 
only rewards balls to successful treatments. Another example is the birth and death 
urn (Ivanova ct al., 2000; Ivanova and Flournoy, 2001). The ternary urn model induces 
diagonal generating matrices. The asymptotic properties of ternary urn model can be 
obtained by embedding the urn process in a branching process. 
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However, there are several serious drawbacks of the classical urn models: (i) it is only 
for binary (multinomial) responses; (ii) it has a predetermined limit that usually does 
not have any formal optimal properties (for example, the randomized play-the-winer rule 
(Wei and Durham, 1978), see Hu and Rosenberger, 2006 for details); (iii) it usually has 
higher variability than other procedures (Hu and Rosenberger, 2003), and therefore is less 
powerful. 

1.2. Objectives and organization of the paper 

In this paper, we develop a more general class of urn models that incorporate immi- 
gration, called the immigrated urn (IMU) model hereafter. Both the GPU model and 
the ternary urn model can be viewed as special cases of the IMU model. An example 
of the IMU model is the the drop-the- loser (DL) rule proposed by Ivanova (2003). But 
the DL rule is limited to binary (multinomial) responses and it has a predetermined limit 
that does not have any formal optimal properties (Hu and Rosenberger, 2006). The IMU 
encompasses a much wider spectrum of urn models and incorporates the immigration 
process, which offers a greater flexibility in the choice of appropriate urn models in appli- 
cations. The main features of the proposed IMU model are: (i) It includes other classes 
of urn model as special cases; (ii) It can be used to target any given allocation proportion 
(optimal allocations in clinical trials, see examples in Section 4); and (iii) in the class of 
IMU models, one can design his own IMU model that yields a smaller variability than 
other urn procedures. 

The exploration of asymptotic properties also helps the reader to evaluate the merits 
of different models in this class. In the literature, asymptotic properties of urn models are 
usually proven by using Athreya and Ney's (1972) technique of embedding the urn process 
in a continuous-time branching process. This technique docs not apply to the IMU due 
to the possible nonhomogcneous immigration process of the IMU. It is worth to point out 
that the asymptotic properties of the GPU model are obtained under specific conditions 
of the generating matrix. However, these conditions are usually not satisfied in framework 
of the IMU models discussed in this article. 

In this paper, we represent the IMU process by an approximated Wiener process and 
then obtain the asymptotic properties of the IMU. The main contributions of this paper 
are: 

(a) to formulate a general class of urn models with the immigration process (IMU 
model); and 

(b) to derive important asymptotic properties, including strong consistency and asymp- 
totic normality of treatment proportions; 

(c) to obtain some useful and desirable IMU models based on the asymptotic properties. 

In Section 2, the IMU process is outlined. Asymptotic properties arc examined in 
Section 3. Illustrative examples that are related to the applications of the IMU are given 
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in Section 4. Concluding remarks are provided in Section 5. Finally, the technical proofs 
are stated in the Appendix. 

2 The immigrated urn model 

Suppose subjects arrive sequentially to be randomized to one of K treatments and 
respond immediately. An IMU model is defined as follows. Consider an urn that contains 
balls of K + 1 types. Balls of types 1,...,K represent treatments, and balls of type are 
the immigration balls. 

Initially, there are .^o,i(> 0) balls of type i, i = . . . ,K. Let Zq = {Zq^q, . . . , Zq^k) 
be the initial urn composition. When a patient arrives, a ball is drawn at random and 
the corresponding treatment will be assigned to this person. Now, right before the m-th 
subject arrives to be randomized to a treatment, let Z^-i = {Z„i-ifl, . . . , Zm-\,K) denote 
the composition of the urn. To avoid a negative likelihood of selecting a treatment, we 
adopt a slight adjustment to .^m-i and let Z^_^- = max(0, Z^-i^j), i = 1,...,K, and 

To randomize the m-th subject, a ball is drawn at random without replacement. The 
probability of selecting a ball of type i is Z^_^ i/\^m-i\i ^ = 0, 1, . . . , K. Here, \Z^_-^ \ = 
J2f=o^m-ij- Therefore, the balls with a negative value in Z^-i will have no chance of 
being selected. 

If the drawn ball is of type (i.e, an immigration ball), no subject is treated and the 
ball is returned to the urn. Furthermore, Am-i = Om-i,i + • • • + a,m-i,K additional balls, 
Om-i,fc(^ 0) of treatment type k, k = 1, . . . ,K are added to the urn. If the immigration 
ball is selected I times before a treatment ball is drawn, the urn composition Z^-i is 
updated to (Z„i_i,o, ^m-i,i + lam-i,i, ■■■ , Zm-i,K + lam-i,K) and the Z'^_-^^ is updated 

to (Zm^ifl, {Zm-1,1 + lam-l,l)~^ , ■ ■ ■ , {Zm-1,K + lo-m-l,K)^)- 

If a treatment ball is drawn (say, of type fc, for some /c = 1, . . . , K), the m-th subject 
is given treatment k and the outcome ^m,k of this subject on treatment k is observed. The 
ball is not replaced. Instead, Dm,kj = Dkj{^yn^k) balls of type j will be added to the urn, 
j = 1, . . . ,K. Djn,ki < signifies the removal of balls. 

With the IMU, the number of immigration balls remains unchanged and a treatment 
ball is dropped when it is drawn. The number of treatment balls that is added to the urn 
depends on 

(a) the value of ajn,k when an immigration ball is drawn from the urn, and 

(b) the value of Dm,kj when a ball of treatment type k is selected. 

Here, a^^kS represents the immigration rates and D^^kj^ represents the adding rules. 
When am,k = 0, we obtain the GPU models if one chooses the adding rule Dm,kj as in 
Section 4.1 in Hu and Rosenberger (2006). When ajn,k = 0) one can obtain the ternary 
urn model by choosing the Dm^kj as in the Section 4.4 in Hu and Rosenberger (2006). 
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For two-treatment case, one may choose am,k = Ij and suitable Dm,kj (see Example 2) to 
obtain the drop-the-loser rule. In general, we can select suitable a^.k and Dj^ j^j to obtain 
the desirable IMU model. Examples can be found in Section 4. 

It is important to note that both a^^k and D^^kj depend on m. This allows both 
the immigration rates and the adding rules to depend on the responses of clinical trial. 
Consequently, it enables us to search for the desirable IMU model that can be used to 
target any given allocation proportion. Since both am,k and Dm,kj depend on m, it is 
impossible to use Atheya and Ney's (1972) technique of embedding the urn process in a 
continuous-time branching process. 

After the treatment assignment of the n-th subject, Nn^k is used to denote the number 
of subjects who are being assigned to treatment k, k = 1, . . . , K . In clinical studies, the 
proportion Nn^k/n, k = 1, . . . , K patients being assigned to various treatments is useful 
information. The distributional properties and statistical behavior ol N^^k/n are discussed 
in Section 3. 



3 Asymptotic Properties 

3.1. Notation and assumptions 

Important asymptotic properties for the immigrated urn model are derived in this 
section. First, we need to introduce some basic notation and the required assumptions. 
Let 9k be an unknown parameter from the distribution of the response ^m,k, and write 
6 = (01, ... , Ok)- Assume that the immigration rates 

am-i,k = ak{dm-i), k = l,...,K, (3.1) 

where afc(-)s are continuous functions, and 0m-i = {Sm-i,i, • • • > Sm-i,k) and 9m-i,k are the 
current estimates oi 6k, k = 1, . . . , K . The immigration process that is defined in ()3.ip has 
two functions. The first is to prevent the possibility of the extinction of a particular type 
of treatment ball. The second is to adjust the treatment allocation proportions according 
to the current estimates of 9. 

Without loss of generality, we assume that the unknown parameter 9k is the mean of the 
outcome ^m.k and we take the sample mean as its estimate. Write = iCm,i, ■ ■ ■ ,S,m,K)- 
For the adding rules, let Dm = {Dm,kfK3 = l,---,-?^), D^m = {DmM^ ■ ■ ■ ' ^■m,kK), 
k = 1, . . . , K , and H = (hkj) = EZ)„. Let 9m-i,k be the sample mean of the outcomes 

n « + Sm-i,k o^ 

7m-l,A: — ^ , „^ , yo.Z) 



(3 + N, 



where Sm-i,k is the sum of the outcomes on treatment k of the previous m — 1 subjects. 
Here, a,/3 > are used to avoid the nonsense case of 0/0. In general, many estimators, 
for example, the MLE, can be written in the form of (j3.2p with Sm-i,k being replaced by 
a sum of functions of the treatment outcomes plus a negligible remainder. 
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Assumption 3.1 Functions afc(-) > are continuous and twice dijferentiahle at 0. 



Assumption 3.2 {(^m.fci ^m.fci) • • • > -Cm,fcA")i ^ ^ 1}) k = 1,...,K, are K sequences of 

1.1. d. random variables with s\x\>,^ E\Dm,kj\'^^^ < oo, and sup^ E\S,m.k\'^^^ < °^ Z'^'" some 
0<6<2, k = l,...,K. Further, assume that Dm,kk ^ —C for some C , k = 1, . . . , K , 
and also, either h/^j > or D^^kj ^ for k ^ j. 

Assumption 3.3 HI' < 1', i.e., Ylf=ihkj < 1 for all k = l,...,K. Here, 1 = 

Assumption 13.31 means that the average number of added balls at each step according 
to the outcome of a treatment is less than the number (one) of dropped balls. Thus, the 
urn is updated mainly by immigration. When the number of added balls exceeds one, the 
total number of balls in the urn will monotonically increase to infinity, and as a result, 
the probability of drawing an immigration ball will monotonically decrease to zero. This 
model is equivalent to the generalized Polya urn model without immigration, which has 
been well studied (c.f., Hu and Rosenberger, 2006). 

For the immigration rates, write Ok = ak{0). Let a = (ai, . . . ,aK), u = a{I — H)^^, 
s = a{I - H)~^l' = J2k=i'^k and v = u/s. Further, denote = Var{D|'^-'}, En = 
Y.k=i^k^k, Si2 = {Coy{Di^kj,ik];i,k = 1, . . . ,K), S22 = diag{\/3r{ii^i, . . . , Var{^i_/^}), 
and 

^ ^ /All Ai2\ ^ / Sii S;i2 diag{v)\ 
Va;2 A22 J \diag{v) S'^a S22 diag{v) ) ' 

Remark 3.1 Notice that hij > for i ^ j. The existence of {I — H)^^ is implied by 
Assumption \3.3l Also, this assumption can be replaced by a more general assumption in 
which there is a vector e = (ei, . . . , ex) such that He' < e' and Ci > 0, i = 1, . . . , K . 

3.2. Main asymptotic results 

Theorem 3.1 Suppose Assumptions \3. ill 3. 31 are satisfied. Denote Nn = {Nn^, ■ ■ ■ , -^n,fc) 
andv = v(6). One can define a 2K -dimensional Wiener processes {W{t),B{t)) such that 

Var{{W{t),B{t))} =tK (3.4) 

and 

Nn-nv = W{n)A+ f ^^dx diag(-)^^^ + o{n^/^-') a.s. (3.5) 
for some e > 0, where A = {I — H)^^{I — Vv), 



^ ' a(0) (/-//)-! 1' de V d9j ' 



Here, 1/v = {l/vi, . . . , 1/vk) 
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The proof of Theorem 13.11 is given in the Appendix. Using Equation (j3.5p . the consis- 
tency property and asymptotic normahty can be derived. 

Corollary 3.1 Under the Assumptions in Theorem \3.1[ 

Nn — nv = OiyJ n log log n) a.s. and ^/n(— - — ^ -^(0, S), (3.6) 

n ' 

where SI = S/j + 25]^ + S/)^ + S'^^, and 

dv{e) 



de 



In particular, if = const, then 

n ' 

if o-m,k = 0-k, k = 1, . . . , K , do not depend on the estimates, then 

n 

Proof. Notice {W{n),J^ ^^dx) is a centered Gaussian vector with 



W{n) = 0{\/n log logn) a.s., 



/ ax = 0(1) + / ax = C'(v"'loglogn) a.s., 

Jo X Je X 

Var{W(n)} = nEii, 

f Bix) \ r' X f\y 
Var < / dx > = 'S22diag{v) / / dxdy = 2n J^22diag{v) 

[Jo X } Jo Jo xy 



and 



Gov < W{n), / dx > = Tii2diag{v) I dx = n Yli2diag{v). 

[ Jo X } Jo X 

3.6p follows from (j3.5p immediately. □ 



Remark 3.2 In practice, the outcomes in clinical trials are not available immediately 
prior to the treatment allocation of the next subject. The parameters can be estimated and 
the urn can be updated only according to the observed responses. In the delayed response 
case, we let ^^(m,/) be the probability that the outcome of the m-th subject on treatment 
k occurs after at least another I subjects arrive. If ;Ufc(m,/) < Cl^'^ for some 7 > 2, we 
can show that the total sum of unobserved outcomes up to the n-th assignment is a high 
order of ^/n and so the conclusion in Theorem \3.1\ remains true. It has been shown that 
the delay machine does not effect the asymptotic properties for many response- adaptive 
designs if the delay decays with a power rate (c.f., Bai, Hu, and Rosenberger, 2002; Hu 
and Zhang, 2004; Zhang et al, 2007). 
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Next, we consider a special case that includes the generalized drop-the- loser rule of 
Zhang et al. (2007) in which Dm is a diagonal matrix. 

Corollary 3.2 Suppose Assumptions \3.1W.S\ are satisfied, Dm,kj = for j ^ k, and 
hk = l- EDi^kk > 0. Write h = {hi,..., hx), 



Vk{6, h) = 77:777- k = l,...,K, 



ak{6)/hk 

V = v{6, h) = {vi{6, h), . . . ,vk(0, h)), and 

dv{e,h) (dvk{0,h) 



■,j,k = l,...,K 



Then, 



dv{e,h) _ ( dvk{e,h) 

~dh~ - V dh, ^ - 1' • • • ' ^ 



V a.s. and -v) ^ NiO,!:), (3.7) 



n n 
where E = Ed + 25]^ + + S'^., 



(dv{e,h)\' ((^li (TlK\dv{0,h) 



an 



---(^)'M%^^.%^)^. 

d (^Dk = V3r{Di^kk}, o-ffc = Var{^i^k}, (^^Dk = Cov{Di^kk, ^k,i} , k = l,2,...,K. 



Proof. It is easy to check that Sn = diag{a'j-)-^vi, . . . , cr'jjj^VK), "^12 = diag{a'^j^^, . . . , u'^dj^) 
S22 = diag{a'^^, . . . , cr|j^), A = diag{l/h){I - I'v), and dv{0, h)/dh = -diag{v)A. The 
results follow from Corollary 13.11 □ 

If Assumption 13.31 is not satisfied and Hi' = 1', the following theorem can be used to 
yield the consistency property of the allocation proportion. 

Theorem 3.2 Suppose Assumptions [01 and \3.S\ are satisfied, and HI' = 1' . Suppose 
further, 1 is a single eigenvalue of H. Then 



Nn — nv = O(ynloglogn) a.s. and Nn — nv = Op{\/n), 
where v is the left eigenvalue vector of H that corresponds to the eigenvalue 1. 

The asymptotic normality of Nn is still unknown when HI' = 1'. The proof of the above 
theorem is given in the Appendix. 
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Hu, Rosenberger and Zhang (2006) studied the variabihty of a randomization proce- 
dure that targets any given allocation proportion. They obtained a lower bound of the 
variability. For the IMU model in this paper, we have following results. When the variance 
of IMU model attains the lower bound, we can use the Cramer-Rao formula to compute 
the variance. In general, we have 

Theorem 3.3 // each Dm,kj is a linear function of a random rjm,k, j = ^, ■ ■ ■ , K . Then, 
we have 



.„,.(*)' (M^. .... |, 

where d = {di, . . . , dx) = (Erji^i, . . . , Eryi^^-). Further, if a(-) = const and Varjr/i fc} is the 

inverse of the Fisher information of dk, then the asymptotic variance- covariance matrix 
of Nn/ \fn attains the following lower bound, 

(S) "^'"^ ((^i^i)"'' • • • ' (^kIk)-') (I^) , (3.10) 
where 1^ is the Fisher information function of parameter dk ■ 

'Pi 

Proof. In fact, if we write d[''^ = ctfc + /3fc^i,A: and K = I • • • | , then 



K 



All = ^ ?;fcVar{r7i,fc}/3fc/3fc 

k=l 

={d^ag{v)K)'d^ag f Y^lil^l, . . . , YflW") d^ag{v)K 



and 



5^12 = {diag{v)K)' diag 



Cov{r/i^i,^i^i} Cov{r/i,i<-,^i,i<-} 



Vi VK 

However, dH/dd^ = diag{lk)K , where 1^ has zero elements except the k-th one which 
is 1. Also, 

' ={1- Hy'^il - H)-' = {I- H)-Uiag{lk)K{I - H)-\ 

It follows that 

dv Ja{I -HY^ddk da{I-H)-'/ddk , .._„wi 
ddk a{I-H)-^V {a{I - H)-^VY ' 
=vdiag{lk)K{I - H)-^{I - I'v) = vdiag{lk)KA, 

i.e., dv/dd = diag{v)KA. (j3.8p and (|3.9p are proved. □ 
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Remark 3.3 In Theorem \3.3\ we assume that the parameter dk is a one- dimensional 
parameter that corresponds to treatment k for simplicity of notation. The results can be 
rewritten to a vector parameter dk without any further assumptions. 



4 Applications 

In this section, we apply the general asymptotic results in Section 3 to selected IMU 
models for illustrative purposes. 

Example 1: The birth and death urn (BDU) 

Ivanova et al. (2000) proposed a birth and death urn (BDU) model to allocate treat- 
ments in clinical trials. In the BDU model, if a ball of treatment k is drawn and a subject 
is assigned to this treatment, the ball is not replaced when the treatment is a failure. 
When the treatment is a success, the ball is replaced and one additional ball of the same 
type is added to the urn. If an immigration ball is drawn, the ball is replaced and in 
addition, one ball of each treatment type is added. The BDU is a special case of the IMU 
with am,k = 1, Dm,kj = for j / k, and 

(a) Dm,kk = 2 if the treatment outcome of the m-th subject on treatment A; is a success, 
and 

(b) Dm,kk = if the treatment outcome of the m-th subject on treatment A; is a failure. 

For this example, the asymptotic variance attains the lower bound. It is easily seen that 
H = diag{2pi, . . . ,2pk), where pk (qk) is the success (failure) probability for treatment 
k, k = 1, . . . , K . By Corollary 13.11 and Theorem 13.31 

Nn,k 1/(1 - 2pk) ^(Nn \^^.T((,^\ 

— - ^Vk = — ^ a.s., Vri v] iV(0, E) 

" Ef=i 1/(1 - 2p,) V ^ J 

if Pk < 1/2, k = 1,. . . ,K, where v = {vi, . . . ,vk), P = (pi, • • • ,Pk) and 



■'D 



( ' diag ( ^^^^ PKQJc \ ^ 
\dp) V ^^1 ' "' VK J dp' 



Example 2: Drop-the-loser (DL) rule and Modified DL (MDL) rule 

The urn process of the DL rule is basically the same as that of the BDU. The major 
difference is that when the treatment is a success, the treatment ball is replaced without 
the addition of extra treatment balls. The DL rule is also a member of the IMU with 

am,k = 1, Dm,kj = for j / k, and 

(a) Dm,kk = 1 if the treatment outcome of the m-th subject on treatment A; is a success, 
and 
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(b) Dm,kk = if the treatment outcome of the m-th subject on treatment is a failure. 

A modified version of the DL rule (MDL) is given as follows. When a treatment ball 
is drawn, this ball is replaced only when the treatment outcome is a success. However, 
when an immigration ball is drawn, instead of adding an equal number of treatment balls 
to the urn, we add Cp^ balls of type A;, k = 1,. . . ^K, where pk is the current estimate 
of the successful probability pk of treatment fc, and C is a constant. With this model, 
more balls are immigrated to treatments with higher success rates, and subsequently, the 
limit proportions will be higher for better treatments. In fact, as shown below, the limit 
proportions are proportional to the odds Pk/lk- For the DL rule, the limit proportions are 
proportional to l/qk (Ivanova, 2003). 

This design can be compared to the generalized Polya urn model that was proposed 
by Bai, Hu, and Shen (2002). For the latter, the asymptotic normality was obtained 
by Zhang, Hu, and Cheung (2006). Unlike the generalized Polya urn models without 
immigration in which the asymptotic normality holds only when a very strict condition 
on eigenvalues of a generating matrix is satisfied (c.f., Bai and Hu, 2005; Janson, 2004; 
Zhang, Hu, and Cheung, 2006), the MDL rule and the IMU in general, as we will show, 
allow asymptotic normality under very mild conditions. 

Regarding the asymptotic variance, it is easily seen that a = {piC, . . . ,pkC) and 
H = diag{pi, . . . ,pk)- The conditions in Corollarv 13.21 are satisfied for all cases with 
< Pk < 1 and k = 1, . . . , K . Therefore, by Corollary 13.21 the limit proportions are 

Vk = ^^ — , fe = l,...,K. 

The asymptotic variance-covariance can be derived by the formulae in Corollary 13.21 in 
which 9 = {pi,...,pk), h = (gi,...,^^), and aj^f, = cr|^ = = PkQk, k = 1,...,K. 
For the two-treatment case, 

^"'1 A n^AT / \ ^ AT in 2n 

— ■ > vi = — — a.s. and \Jn{Nni 'n — vi)^N{\),a), 

n pi/qi+P2/q2 

where cj^ = (?i(?2bi(l + ^i) +^2(1 + 1i)]l{P2qi + Piq2Y ■ However, the lower bound of the 
asymptotic variance is 92(^1 +P2)/(?'2'i'i +Pi'i'2)^- 

Example 3: Optimal proportions for two treatments with dichotomous outcomes. 

To compare two treatments with dichotomous outcomes, success or failure, Rosneberger 
et al. (2001) suggested an optimal proportion vi = a/Pi/(a/Pi + a/P2)) where (^2) is the 
successful probability of treatment 1 (2). To target this proportion, Zhang et al. (2007) 
proposed a generalized drop-the- loser urn, a special case of IMU with a^^k = C^Jp^-, 
k = 1,2, and Dm = 0. By Corollary 13.11 or Corollary 13.21 the asymptotic variance of the 
(normalized) sample proportion is 

1 f P2qi ^ Piq2 
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which is twice the lower bound. Zhang ct al. (2006) propose to use the GPU without im- 
migration to target this proportion (c.f., their Example 2). The corresponding asymptotic 
variance is 



which is much larger. 

5 Conclusions 

In this paper, we have generated a general class of urn models that incorporates im- 
migration. As illustrated in the examples, the IMU class comprises of many useful urn 
models. For explicatory purposes, we focus our discussion of these applications to the 
clinical environment, but these urn models can be applied to other areas for which the 
setting is deemed appropriate. 

Important asymptotic properties are derived in this paper for the IMU class. With 
the asymptotic normality formula, the models can be evaluated in terms of the distribu- 
tions of treatment allocation proportions. Under very mild conditions, the IMU models 
always yield relatively small asymptotic variances. In many cases, the asymptotic variance 
attains the lower bound. Thus, the IMU model usually has smaller variability than the 
corresponding urn model. 

When the IMU models are applied in clinical trials, the responses may not be available 
immediately and the population could be heterogenous. There is no logistical difficulty 
incorporating delayed responses into the IMU models. One can update the urn when 
responses become available. A moderate delay in response (see Hu and Zhang, 2004) 
will not affect the asymptotic properties of the IMU. One can modify the proof in the 
Appendix to incorporate delayed responses. It can also be shown that the IMU model 
can accommodate heterogeneity under the conditions that are discussed in Bai and Hu 



In this article, we have focused on the asymptotic properties of the IMU itself. It is 

important to assess the validity of the statistical inferences after using the IMU models. 
By applying the results of Hu, Rosenberger, and Zhang (2006), Theorem 3.1 and Corollary 
3.1 ensure that the estimators have the same asymptotic properties as they would if a fixed 
design were used for the same allocation proportion. This provides a solid foundation for 
both the IMU model and its related statistical inferences. The applications of IMU models 
have been discussed mainly in the clinical context, but the IMU could play an important 
role in other areas because of its desirable properties. Clearly, the application of the IMU 
in other areas requires further research effort. 




(2005). 



12 



Appendix. Proofs 



We prove Theorem 13.11 first. Recall that Zm-i = (-^m-i.Oi Zm-i,i, ■ ■ ■ , Zm~i,K) are the 
numbers of balls when the m-th subject arrives to be randomized, -^^-i ~ i^m-i c ^m~i i' 
^m-i,K) non-negative numbers, and = ^m_i,o + ^m~i,i + •■■ + 

^m-iK- Write Zm-i = {Zm-i,i, ■ ■ ■ , Zm-i,K)- Becausc every immigration ball is re- 
placed, Z^_^ Q = Zm-1,0 = Zofi for all m. Let Xm be the result of the m-th assign- 
ment, where Xm,k = 1 if the m-th subject is assigned to treatment k and other- 
wise, k = 1,. . . ,K. Then, Nn = (iV„,i, • • • , A'n.x) = Y.m=i ^m- Further, we denote 
o-m = (om,!, • • • ,o.m,K), and Um to be the number of draws of type balls between the 
(m — l)-th assignment and the m-th assignment. 

Notice that between the (m — l)-th assignment and the m-th assignment, we have 
drawn Um balls of type 0. Accordingly, we have added am-i,kUm balls of type k to the 
urn. However, when a ball of type k is drawn, this ball is not replaced and another Dm,kj 
balls of type j are added to the urn. So, the change in the number of balls after the m-th 
assignment is 

Zm ~ Zyfi—i = dm—lUm + Xjfil^Dm ~ I)- (-'^•l) 

It follows that 

71 n 



Zn — Zq — ^ am~lUm + ^ Xm{Dm — I) 
m=l m=l 
n n 

-- ^ flm-l^m - Nn{I - ii") + ^ Xjn{Dm - 

m=l m=l 
n 

--aNnfl + (a^-i - a)ura - Nn{I -H)+ Mn, (A.2) 



m=l 



where A''„^o = Z^m=i ""m is total number of draws of type balls after the n-th assignment, 
and Mn = Z]m=i ^m{Dm " ^[Dm]) IS a martingale. 

To prove Theorem 13.11 we need two lemmas the proofs of which are stated later. 

Lemma A. 1 Suppose Assumptions \3. 1^3.3\ are satisfied. Then, 

Zn,k = o{n^/'^-^'') a.s., k = l,...,K (A.3) 

/orO<5o<i-2T5- 

Lemma A.2 Suppose Assumptions [3n^3.3\ are satisfied. Then, 



Nnfl = n/s + 0(v^n log log n) a.s., (A. 4) 



Nn,k = nvk + 0(\/nloglogn) a.s., k = l,...,K, (A.5) 
where s = a{I — H)^^l' . Also, for each k = I, . . . , K , 

6n,k Ok a.s. (A.6) 
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and 

Kk-Ok = ^ + o{n-^"-''^) a.s., (A.7) 
nvk 

where Qn,k = Z]m=i Xm,k{im,k - E(,m,k) is a martingale and Qn = (QnA, Qn,K)- 

Now we begin the proof of Theorem 13. li Consider the 2A'-dimensional martingale 
{{Mn, Qn),An;n > 1}, where An = o"(^i, • • .,Xn,^i, ■ ■ ■ ,^n+i)- According to ()A.5p we 
have 

n K 

E[(AMO'AMi|A-i] = Y,Nn,k^k 

i=l k=l 

= nXlii + 0(\/nloglogn) a.s., (A.8) 

n 

5^E[(AQi)'AQi|A-i] = ^22diag{Nn) 
1=1 

= nA22 + 0(\/n log logn) a.s., (A. 9) 

n 

^E[(AMO'AQ,|A-i] = ^i2diag{Nn) 
1=1 

= nAi2 + 0(\/n log log re) a.s. (A. 10) 

By Corollary 1.1 of Zhang (2004), we can define the 2Er- two-dimensional Wiener processes 
{W{t),B[t)) with variance-covariance matrix A such that for some e > 0, 

Mn = W{n) + o{v}'^'') a.s., Q„ = B{n) + o{n^/^'') a.s. (A.ll) 

Without loss of generality, we assume e < 5q, where 5q is defined as it is in Lemma I A.ll 
Next, we need to show that {W{t), B{t)) satisfies (|3.5p . Combining (|A.2p and (jA.Sp yields 

n 

Nn{I -H)- aNnfl = Mn + Y{am- a)um + o(reV2-5o) a.s. (A.12) 

m=l 

Recall A = (J-i3")-i(/-l't>), v = a{I-H)-^ /{a{I-H)-^l') and notice Nnl' = re. 
According to (|A.12p . 

n 

Ar„-reti = (M„+ ^(a„_i-a)n„)A + o(rei/2-5o) q,.s. (A.13) 

m=l 

For a™ - a, due to (|A77|) and fOT]) . 

an,- a ={dn, - 0)^^ + O(||0™ - 0f ) 

-^'"-d^a5(i)^ + o(m-V2-^o) (A.14) 



m V do 

B{m) ^. ,l.da{e) . _i,2-,. 
--^^diag{-)^-^ + o{m ^'^ 
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Note that immigration happens only when a type ball is drawn. Let Tm be the total 
number of draws when the m-th type ball is drawn. At this time, Tm — "m subjects 
are assigned and the {Tm — m + l)-th subject arrives to be randomized. So, we add 
'^(Tm-m+i)-i,k balls of type k to the urn, k = 1, . . . , K. It follows that 

n Nn.o 

j=l m=l 

n Nn,0 

i.e., (ftm-i - a)um = {o-Tm-m - a)- 

m=l m=l 

It is easily seen that Tm = min{n : A^^^o ^ + Due to ()A.4p . 



Tm — m = min{n : A„,o ^ = sm + ©(y^mlog logm) a.s. 

It follows that 



fJ"r.. — m Oi 



Bijm-rn) A.da{e) -l/2-e^ 



B{sm) A.da{9) 



diaai-) an +o{m 



sm V do 

Noticing (|A.4p . we conclude that 

N„,o 



a.s. 



E, , f B{sm) ,l.da{6) _ 

m=l m=l ^ 



1/2-6- 



Jo sx ^v' od 







—^dxdiagi-) wtt^ + oin^'^-') a.s. A.15) 

X ^v' s oO 

However, it is easily checked that 

1 da{e) dv{e) 



s 86 de ' 

Combining ()A.lip - ()A.16P finishes the proof of ([33]) . □ 



(A.16) 



Now, we begin the proofs of Lemmas lA.ll and IA.2I We need three more lemmas. 
Lemma A.S Under Assumption \3.S\ we have 

Z .=0(1) a.s., k = l,...,K. 
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Proof. Notice that the balls with a negative number have no chance to be drawn. If we 
let Sn = max{l < j < n : Zj^^ > 0}, then according to ()A.ip . 

Zn,k ^■^n-l,fc + Xn,jDn±j > ■ ■ ■ 

n 

>^5„,A: + ^5„+i,fc(-C's„+l,fcfc - 1) + ^ ^ Xm,jDm,kj 

j^k m=S„+l 

n 

> — C — 1 + y ^ y ^ XmjDjn i^j. 
j^k m=S„+l 

If D„i,kj > 0, then ^J^=s„+i ^m,jDm,kj > 0. When hkj > 0, it is easily seen that 

n n n 

m=Sn+l m=Sn+l m=Sn+l 

+00 a.s. 

on {^m=s„+i -^"m-J ~^ °°}- '^'^^ lemma is now proved. □ 

Lemma A. 4 Let Tn = . . . , Zi,. . . , Zn) be the history sigma field, and recall 

= SfeLi '^m,k- Suppose Assumption \3.^ is satisfied. Then, A := min^ Am > implies 

K +17 

EK\Tn-i]<Op(^{Y,Zn-i,ky/Ay ^^a.s.,yp>l, (A.17) 

k=l \^n~\\ 

where Cp > is a random variable that is a function of Zq^q and minm^m- Particularly, 



mmAm>0 implies E[u^\Tn-i] = 0{1) a.s. (A. 18) 

m 

Proof. Write A = minm^m- The event {un = 1} means that when the n-th subject 
is assigned, we have drawn / + 1 balls continuously in which the first I balls is of type 
and the last one is not. So, for / = 1, 2, . . ., 

p( —/IT \ _ -^0,0 TT ^Ofi f-. Zpfi 

Obviously, P(u„ = l\Tn-i) < Zo,o/\Z^-i\i I ^ 1- Notice 

K K 
k=l k=l 

It follows that ^ > and J2^=i Zn-i,k > —LA imply 
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where cq > depends only on A and Zq^q. So 

/=1 l^n-ll i=L+l l^n-ll l^n-ll 

Taking L = [(EfcLi -^n-i,fc)"M] + 1 completes the proof of (lATTTj) . (lAl8|) follows from 
()A.17P and Lemma El □ 

Lemma A. 5 Suppose Assu'mptions [3n^3.^ are satisfied. Then 

mmam,fc > and maxam,fc < oo a.s. (A. 20) 

m,k ' m,k ' 

Proof. It suffices to show that 

Nn,k ^ oo implies ^fc o.s., k = l,...,K. (A. 21) 

In fact, if ()A.21|1 is true, then ak{y) > for any y on 

K 

d{Om; m = 1, 2, . . .} = (^{9k,em,k; m = 1, 2, . . .}. 

k=l 

By the continuity of afc(-), < mium^k o.k{(^m) < Om,fc < maxm,fc flfcC^m) < oo a.s. ()A.20p 
is satisfied. Now, notice {Qn,k,-^n} is a martingale with 

n 

^ E[(AQ„,fc)Vm~i] = A^n,feVar{ei,fc} < CA^„,fe. 

m=l 

By Theorem 3.3.10 of Stout (1974) (where /(x) = x), 

Nn k —>■ OO implies ^"'^ — > a.s. (A. 22) 

By noticing O^^k = {a + N^^kOk + Qn,k)/{Nn,k + P), ^IM is proved. □ 
Proof of Lemma lA.ll By Lemma IA.5| 

A =: min^m > and A =: max^^ < oo (A. 23) 

m m 

Notice Z„l' = Ef=i^n-i,fc- By (lATTll and Lemma D E[un\J^n-i] < Co^o,o/|-^+_i|. 
So, according to (jA.ip or ()A.2p . we have 

=Z„_il' + UnAn-l - Xn{I " H)l' + AM„l' 

<Z„_il' + An-lE[Un\J'n-l] - h + A„_i(u„ - E[u„|.^„_i]) + AM^l' 
<Z„_il' + Co:4-§^ -h + AUn 

<Z„_il' + AC/„ - /i /2, if Zn-il' > 2CoAZo^o/h, (A.24) 



17 



where h = mmfc(l - Y.f=i hkj) > 0. Here, f/„ = I]m=i ^m-i(nm - E[um\J^n-i]) + Mnl' 
is a real martingale. Let Sn = max{l < j < n : Zjl' < 2C()AZQ^Q/h }, where max(0) = 0. 
Then, according to (|A.24p . 

Z„l'<Z„_il' + A[7„-/i/2<... 
<Zs„r + AC/5„+i + . . . + AC/„ - (n - /2 

<|Zo| V (2CoAZo,o//i) + C/n - - (n - /2. (A.25) 
For the martingale {Un,^n',n = 1,2,...}, we have 

E[(AC/„)Vn-i] < C + Cmax^2 ^O(l), 



E[|AC/„|2+5|.F„_i] < C + Cmax ^2+5 ^ q^^^^ 

j ■' 

due to Assumption 13.21 and (jA.lSp . Accordingly, we can show that 



Un = 0(\/nloglog n) a.s. (A.26) 

and 

max \Un-[./T[Tni^+m - ^n-[y;n^l = logn) a.s. (A.27) 

m< V n log n 

If n — > \/n logn, then for n large enough 



Un - Us„ - (n - Sn)h /2 < 0(Vri log logn) - h^/nlogn/2 < 
due to ()A.26p . Notice n > Sn- If n — 5„ < ^n logn, then 

Un - Us„ - (n - Sn)h /2 <2 max |?7„_^^^n^]+^ - f^„„[V?n^ I 

m<Vnlogn 
1 

=o(n2+* logn) a.s. 

by (|A:271) . It follows that Ef=i < o(ni/2-5o) ^.s. due to (1X25]) . However, Z"^ = 
0(1) a.s. by Lemma fA. 31 (|A.3p is proved. □ 

Proof of Lemma ET2i Recah Qn,k = Ylm=i ^m,kiCm,k -^k) k = 1,... ,K, and both 
{Mn^fc,^„;n > 1} and {Qn,k,-An',n > 1} are martingales. According to the law of the 
iterated logarithm for martingales, we have 



Mn,k = 0(-\/n log logn) and Qn,k = 0{\/n log logn) a.s. (A. 28) 

However, for each k = 1, . . . ,K, 

Qn,k -Gk = — TT —3— a.s. (A.29) 

Nn,k + P 
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()A.12p remains true by Lemmas lA.ll By ()A.12p and ()A.28P we have 

n 

Nn{I - H) = ^ am-iUm + o{n) a.s. (A. 30) 



m=l 

Notice HI' <1'. It follows that 



n = Nnl' < ^ Am-iUm/h + o(n) < [Ka/li)Nnfl + o{n),a.s. 
where h = minfc(l — Ylif=i ^kj) > 0. That is, 



liminf ^ > 1^ > a.s. (A.31) 



Notice hkj >0,kj^j. Then, by fOOl) and fOT]) . 

(1 — /ifcfc) liminf — ^ > liminf — Um k^m o ^ aliminf — ^ > a.s.. 

m=l 

So, liminf„^oo A^„^,fc/n > a.s., which, together with ()A.29P and ()A.28p . implies 



^n. - e, = 0( ^-^ + ^^^^ l = 0(J'^^^) ^ a.s. 



n / \ V n 
()A.6P is proved and also 



^ ^ / /lo£loE?n\ 

Om,fc-afc = afe(6'm)-afc(6') = 0(||6>„-6'||)=0(y^-^j a.s. (A.32) 

Notice that I]m=i("^m-i,fc ~ <^k){um - ^[um\^m~i]) is a martingale and 

■(am-l,fc - ak){Um - E[Um|.?>n-l])\2 



771=1 



m— 1 



< > E u„ J^j^_i < C > 5 — < cx) a.s. 

z — / .z — / y^^z 
m=l m=l 

due to (jA.lSP and ()A.32p . By the law of large numbers, it follows that 

n n 

(«m-l,fc - ak)Um = ^ {am-l,k " aA:)E[nm|^m-l] + o{y/n) 



m=l m=l 
n 



= 0(^i^^^^)0(l) + o(V^) = O(Vnloglogn) a.s. (A.33) 

m=l 

by (|Xl8]l and (fXMI) again. Combining ^02]) . (fX28D . and (fAlM]) yields 



Nn - Nn,oa{I - H) 1 = O(Vnloglogn) a.s., 
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which, together with A?"„l' = n, imphes ()A.4p and (jA.Sp . Then combining (jA.Sp . ()A.28p . 
and (|A:29|) yields 

nf + C ( V log log n) nvt 

()A.7p is proved, and the proof of Theorem 13.11 is completed. □ 

Proof of Theorem 13.21 Suppose Assumptions 13.11 and 13.21 are satisfied. Notice 
HI' = 1'. Similar to (1X241) . 

Znl' =Zn-ll' + UnAn-l + AM„l' < + CqA-^^ + AC/„ 

<Z„_il' + CoA/^ + A;7„,, if Zn-il' > ^o,o\/^- 

It follows that 

Znl' <Zsj! + A;7s„+i + . . . + AC/„ + Co:4(n - 

<2Co A Vra + f/„ - f/5„ < 2Col^/^ + 2 max | C/^ | , 

m<7i 

where Sn = max{l < j < n : ZjV < Zq^/^/u } and max(0) = 0. Hence, 



Zn = 0{y^nlog logn) a.s. and = Op{\/n), 

by the properties of a martingale. So, by (IA.2P and the law of the iterated logarithm of 
martingales, it follows that 

n 

Nn{I - H) =Mn + a.m^iU^m " + Zq 

m=l 

n 

= am-iUm + 0(\/n log log re) a.s. 



m=l 



Multiplying 1' yields X^m=i '^m-^m-i = 0(\/re log logn) a.s., and then A''„^o = 0{-\/n log logn) 
a.s. and X]m=i*^m-i^m = 0(V?^ log logn) a.s. by ()A.20p . So, 



{Nn - nv){I -{H- I'v)) = Nn{I -H) = 0(Vn log log re) a.s. 



It follows that Nn — nv = 0{yjn log log re) a.s. because (/ — [H — I'v)) is reversible. The 
proof of Nn — nv = Op{^/n) is similar. □ 
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